Matrix Learning in Learning Vector Quantization

We propose a new matrix learning scheme to extend Generalized Relevance Learning Vector Quantization (GRLVQ), an efficient prototype-based classification algorithm. By introducing a full matrix of relevance factors in the distance measure, correlations between different features and their importance for the classification scheme can be taken into account and automated, general metric adaptation takes place during training. In comparison to the weighted euclidean metric used for GRLVQ, a full matrix is more powerful to represent the internal structure of the data appropriately. Interestingly, large margin generalization bounds can be transfered to the case of a full matrix such that bounds which are independent of the input dimensionality and the number of parameters arise. This also holds for local metrics attached to each prototype. The algorithm is tested and compared to GLVQ without metric adaptation [16] and GRLVQ with diagonal relevance factors using an artificial dataset and the image segmentation data from the UCI repository [15].